Effective Subspace Clustering with Dimension Pairing in the Presence of High Levels of Noise

Attempts at clustering large and high dimensional data have been made with a focus on scalability. While still inefficient for more complex problems, the effectiveness is also questionable because data becomes very sparse in a high dimensional space. If clusters exist in the data, they tend to remain hidden in some unidentified sub-spaces. So far, the few solutions to this problem have not been able to handle high levels of noise and are often inefficient. Most clustering solutions also require fine-tuning of parameter settings, which are difficult or impossible to set in advance especially by a novice user. We propose a new method, MAXCLUS, which first identifies sub-spaces where clusters could be located then pinpoints the clusters in each sub-space. MAXCLUS has proved robust even with high levels of noise and is very efficient and accurate with little or no parameter adjustment on a wide range of problems outperforming existing approaches.